All-to-all communication is collective communication in which each node that is connected to a network performs transmission and reception of communication data for all of the nodes (including the each node) that are connected to the network. In the all-to-all communication, an appropriate communication algorithm varies depending on the size of communication data or a network topology. For example, when the size of the communication data is smaller than the size of general communication data, the communication delay and the number of communications become bottlenecked. Therefore, a communication algorithm is selected by which the communication delay hardly occurs, and the number of communications is reduced. In addition, the size of the communication data is larger than the size of general communication data, the width of the communication bandwidth becomes bottlenecked. Therefore, a communication algorithm is selected by which the width of the communication bandwidth is widely obtained.
The following technology of all-to-all communication is known. For example, in each computer in a parallel computer system, the speed-up of the communication is achieved by determining a communicating party at each phase so that the communication standby state does not occur.
In addition, the following technology of all-to-all communication is also known. For example, in order to avoid the occurrence of Head-of-Line (HoL) blocking in a network switch of a multi-core personal computer (PC) cluster, a 2-Level Ring algorithm is employed instead of a Ring algorithm. As a result, the HoL blocking is avoided, and a reduction in the utilization efficiency of the network is avoided when the all-to-all communication is performed.
The all-to-all communication is also performed in a fat-tree type system in which a network topology having a tree-shaped structure, called a fat-tree, is employed. The fat-tree type system is, for example, a system in which network switches are connected to each other at multiple-stages, and a plurality of highest-order network switches are provided, as illustrated in FIG. 1. In the example of FIG. 1, Leaf switches 1 to 3 are arranged at the positions of the end of the fat-tree, and Spine switches 1 to 3 are arranged as higher-order network switches than the Leaf switches 1 to 3. Nodes are connected to each of the Leaf switches 1 to 3. For example, nodes 0 to 2 are connected to the Leaf switch 1, and nodes 3 to 5 are connected to the Leaf switch 2, and nodes 6 to 8 are connected to the Leaf switch 3. It is assumed that communication data that is exchanged between the nodes passes through one of the Spine switches 1 to 3.
In the system of such a fat-tree type, the influence range is increased when a failure occurs in the network switch unless a communication algorithm is appropriate for the all-to-all communication. However, in the above-described related arts, such a problem is not considered.
The related art is discussed in Japanese Laid-open Patent Publication No. 11-110362.
In addition, the related art is also discussed in Naruse Akira, Nakashima Kohta, Sumimoto Shinji, and Kumon Kouichi, “A Proposal and Evaluation of All-to-all Algorithm for Multi-core PC Cluster Systems”, Symposium on Advanced Computing Systems and Infrastructures (SACSIS) 2010, pp. 311-318, May 2010.